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IN THE CLATM ^- 

The following is a complete listing of the claims, and replaces all earlier 
listings and all earlier versions. Specifically, Claim 33, the only claim amended, has been 
amended to reinsert the phrase "an improvement" into the preamble of the claim. 




Claim 1 (original): A method of processing input data defining (i) the 
positions of features in a sequence of images of at least one object which represent features 
on the object, (ii) an estimate of a respective camera projection for each image defining the 
projection of points on the object into the image, and (iii) 3D feature points comprising 
estimates of the positions in three-dimensions of features on the object represented by the 
features in the images, to generate output data defining further estimates of the camera 
projections, the method comprising processing respective subsets of the images in the 
sequence by: 

calculating an error for the camera projections of the images in the 
subset by projecting selected 3D feature points into the images in the subset using the 
camera projections of the images in the subset and determining the distance between the 
positions of the projected points and the positions of the corresponding features in the 
images; and 

calculating changed positions for at least some of the selected 3D 
feature points and changed camera projections for the images in the subset by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
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points and changed camera projections for the images in the subset which reduce the 
calculated error, 

wherein the respective subsets of images are selected sequentially 
from the sequence so that each image in the sequence is processed in a subset at least once. 

Claim 2 (original): A method according to claim 1, wherein the changed 
positions for the 3D feature points and the changed camera positions are calculated by 
reducing the calculated error by performing processing in accordance with a minimisation 
procedure to change iteratively the positions of at least some of the selected 3D feature 
points and the camera projections of the images in the subset and to calculate an updated 
error by projecting the 3D feature points from their changed positions into the images in the 
subset using the changed camera projections of the images in the subset, and determining 
the distance between the positions of the projected points and the positions of the 
corresponding features in the images. 

Claim 3 (original): A method according to claim 2, wherein the calculated 
error is reduced by performing processing in accordance with a non-linear least squares 
minimisation procedure. 

Claim 4 (original): A method according to claim 3, wherein the calculated 
error is reduced by performing processing in accordance with a Levenberg-Marquardt 
minimisation procedure. 
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Claim 5 (original): A method according to claim 1, wherein the changed 
positions for the 3D feature points and the changed camera projections are calculated by 
decomposing a matrix representing the positions in the images in the subset of the features 
which correspond to the 3D feature points to be changed into the product of a first matrix 
representing the changed camera projections of the images in the subset and a second 
matrix representing the changed positions of the 3D feature points. 

Claim 6 (original): A method according to claim 1, wherein the selected 3D 
feature points used to calculate an error comprise every 3D feature point which corresponds 
to a feature having a measured position in at least one of the images in the subset being 
processed. 

Claim 7 (original): A method according to claim 1, wherein the 3D feature 
points for which changed positions are calculated comprise every 3D feature point which 
has a measured position in at least one of the images in the subset being processed but no 
measured position in any of the other images in the sequence. 

Claim 8 (original): A method according to claim 1, wherein each respective 
subset contains the same number of images. 

Claim 9 (original): A method according to claim 1, wherein the number of 
images in a subset is set in dependence upon the number of features in the images having a 
position defined in the data to be processed. 
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Claim 1 0 (original): A method according to claim 1 , wherein the respective 
subsets of images are selected sequentially from the sequence with an overlap, 



Claim 1 1 (original): A method according to claim 1, further comprising the 
step of generating a signal conveying the further estimates of the camera projections. 

Claim 1 2 (original): A method according to claim 1 1 , further comprising 
the step of recording the signal either directly or indirectly. 



Claim 13 (original): A method according to claim 1, further comprising the 
step of processing image data defining the images in the sequence to generate the input 
data. 

Claim 14 (original): A method according to claim 1, further comprising the 
step of using the further estimates of the camera projections to generate data defining a 3D 
computer model of the scene in the images. 

Claim 15 (original): A method according to claim 14, further comprising 
the step of generating a signal conveying the 3D computer model. 

Claim 16 (original): A method according to claim 15, further comprising 
the step of recording the signal either directly or indirectly. 
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Claim 17 (previously presented): An apparatus for processing input data 



defining (i) the positions of features in a sequence of images of at least one object which 
represent features on the object, (ii) an estimate of a respective camera projection for each 
image defining the projection of points on the object into the image, and (iii) 3D feature 
points comprising estimates of the positions in three-dimensions of features on the object 
represented by the features in the images, to generate output data defining further estimates 
of the camera projections, comprising a processor for processing respective subsets of the 
images in the sequence by: 



subset by projecting selected 3D feature points into the images in the subset using the 
camera projections of the images in the subset and detennining the distance between the 
positions of the projected points and the positions of the corresponding features in the 
images; and 



feature points and changed camera projections for the images in the subset by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
points and changed camera projections for the images in the subset which reduce the 
calculated error, 

wherein the processor is arranged to process respective subsets of 
images from the sequence sequentially such that each image in the sequence is processed 
a subset at least once. 




calculating an error for the camera projections of the images in the 



calculating changed positions for at least some of the selected 3D 



-6- 



PA6E 9/34 ' RCVD AT 51612004 4:38:30 PM [Eastern Daylight Time] ' SVR:USPTO-EFXRF-1/0 ' DN1S:8729306 ' CSID:1212 218 4561 ' DURATION (mm-ss):0844 



05/06/2004 16:40 FAX 1212 2l^?561 FITPATRICK N.V. @]010 



Claim 18 (previously presented): An apparatus according to claim 17, 
wherein the processor is arranged to calculate the changed positions for the 3D feature 
points and the changed camera positions by reducing the calculated error by performing 
processing in accordance with a minimisation procedure to change iteratively the positions 
of at least some of the selected 3D feature points and the camera projections of the images 
in the subset and to calculate an updated error by projecting the 3D feature points from their 
changed positions into the images in the subset using the changed camera projections of the 
images in the subset, and determining the distance between the positions of the projected 
points and the positions of the corresponding features in the images. 

Claim 19 (previously presented): An apparatus according to claim 1 8, 
wherein the processor is arranged to reduce the calculated error by performing processing in 
accordance with a non-linear least squares minimisation procedure. 

Claim 20 (previously presented): An apparatus according to claim 19, 
wherein the processor is arranged to reduce the calculated error by performing processing in 
accordance with a Levenberg-Marquardt minimisation procedure. 

Claim 21 (previously presented): An apparatus according to claim 17, 
wherein the processor is arranged to calculate the changed positions for the 3D feature 
points and the changed camera projections by decomposing a matrix representing the 
positions in the images in the subset of the features which correspond to the 3D feature 
points to be changed into the product of a first matrix representing the changed camera 
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projections of the images in the subset and a second matrix representing the changed 
positions of the 3D feature points, 

Claim 22 (previously presented): An apparatus according to claim 1 7, 
wherein the processor is arranged to perform processing so that the selected 3D feature 
points used to calculate an error comprise every 3D feature point which corresponds to a 
feature having a measured position in at least one of the images in the subset being 
processed. 

Claim 23 (previously presented): An apparatus according to claim 17, 
wherein the processor is arranged to perform processing so that the 3D feature points for 
which changed positions are calculated comprise every 3D feature point which has a 
measured position in at least one of the images in the subset being processed but no 
measured position in any of the other images in the sequence. 

Claim 24 (previously presented): An apparatus according to claim 17, 
wherein the processor is arranged to perform processing so that each respective subset 
contains the same number of images. 

Claim 25 (previously presented); An apparatus according to claim 17, 
wherein the processor is arranged to perform processing so that the number of images in a 
subset is set in dependence upon the number of features in the images having a position 
defined in the data to be processed. 
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Claim 26 (previously presented): An apparatus according to claim 17, 
wherein the processor is arranged to perform processing so that the respective subsets of 
images are selected sequentially from the sequence with an overlap, 

Claim 27 (previously presented): An apparatus according to claim 17, 
further comprising an input data generator for processing image data defining the images in 
the sequence to generate the input data. 

Claim 28 (previously presented): An apparatus according to claim 1 7, 
further comprising a 3D computer model data generator for using the farther estimates of 
the camera projections to generate data defining a 3D computer model of the scene in the 
images. 

Claim 29 (original): A storage device storing instructions for causing a 
programmable processing apparatus to become operable to perform a method as. set out in 
at least one of claims 1 to 16. 

Claim 30 (original): A signal conveying instructions for causing a 
programmable processing apparatus to become operable to perform a method as set out in 
at least one of claims 1 to 16. 

Claim 3 1 (previously presented); In an image processing method for 
processing image data defining a sequence of images of at least one object to generate a 
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three-dimensional computer model of the object by matching features in the images, using 
the matching features to determine camera projections defining the projection of the object 
surface into the images, and using the matching features and the calculated camera 
projections to generate data defining the three-dimensional computer model of the object, 
performing the step of using the matching features to determine the camera projections by 
processing data defining (i) the positions of features in a sequence of Images of the object 
which represent features on the object, (ii) an estimate of a respective camera projection for 
each image defining the projection of the object into the image, and (iii) 3D feature points 
comprising estimates of the positions in three-dimensions of the features on the object 
represented by the features in the images, to generate data defining further estimates of the 
camera projections, comprising processing respective subsets of the images in the sequence 
such that: 

an error is calculated for the camera projections of the images in the 
subset by projecting selected 3D feature points into the images in the subset using the 
camera proj ections of the images in the subset to give projected points and determining the 
distance between the positions of the projected points and the positions of the 
corresponding features in the images; and 

changed positions for at least some of the selected 3D feature points 
and changed camera projections for the images in the subset are calculated by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
points and changed camera projections for the images in the subset which reduce the 
calculated error, 
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wherein the respective subsets of images are selected sequentially 
from the sequence so that each image in the sequence is processed in a subset at least once. 

Claim 32 (previously presented): In an image processing apparatus for 
processing image data defining a sequence of images of at least one object to generate a 
three-dimensional computer model of the object by matching features in the images, using 
the matching features to determine camera projections defining the projection of the object 
\ surface into the images, and using the matching features and the calculated camera 

\ projections to generate data defining the three-dimensional computer model of the object, a 
J method of performing the processing to determine the camera projections by processing 
/ data defining (i) the positions of features in a sequence of images of at least the object 

which represent features on the object, (ii) an estimate of a respective camera projection for 
each image defining the projection of the object surface into the image, and (iii) 3D feature 
points comprising estimates of the positions in three-dimensions of the features on the 
object represented by the features in the images, to generate data defining improved 
estimates of the camera projections, the processing comprising processing respective 
subsets of the images in the sequence by; 

calculating an error for the camera projections of the images in the 
subset by projecting selected 3D feature points into the images in the subset using the 
camera projections of the images in the subset to give projected points and determining the 
distance between the positions of the projected points and the positions of the 
corresponding features in the images; and 
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calculating changed positions for at least some of the selected 3D 



feature points and changed camera projections for the images in the subset by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
points and changed camera projections for the images in the subset which reduce the 
calculated error, 



from the sequence so that each image in the sequence is processed in a subset at least once. 



processor for processing image data defining a sequence of images of at least one object to 
generate a three-dimensional computer model of the object by matching features in the 
images, using the matching features to determine camera projections defining the projection 
of the object surface into the images, and using the matching features and the calculated 
camera projections to generate data defining the three-dimensional computer model of the 
object, an improvement wherein the processor is arranged to use the matching features to 
determine the camera projections by processing data defining (i) the positions of features in 
a sequence of images of the object which represent features on the object, (ii) an estimate of 
a respective camera projection for each image defining the projection of the object into the 
image, and (Hi) 3D feature points comprising estimates of the positions in three-dimensions 
of the features on the object represented by the features in the images, to generate data 
defining further estimates of the camera projections, the processing performed by the 
processor comprising processing respective subsets of the images in the sequence such that; 
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an error is calculated for the camera projections of the images in the 
subset by projecting selected 3D feature points into the images in the subset using the 
camera projections of the images in the subset to give projected points and determining the 
distance between the positions of the projected points and the positions of the 
corresponding features; in the images; and 

changed positions for at least some of the selected 3D feature points 
and changed camera projections for the images in the subset arc calculated by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
points and changed camera projections for the images in the subset which reduce the 
calculated error, 

wherein the respective subsets of images are selected sequentially 
from the sequence so that each image in the sequence is processed in a subset at least once. 

Claim 34 (previously presented); An apparatus for processing input data 
defining (i) the positions of features in a sequence of images of at least one object which 
represent features on the object, (ii) an estimate of a respective camera projection for each 
image defining the projection of points on the object into the image, and (iii) 3D feature 
points comprising estimates of the positions in three-dimensions of features on the object 
represented by the features in the images, to generate output data defining further estimates 
of the camera projections, comprising processing means for processing respective subsets 
of the images in the sequence by: 
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calculating an error for the camera projections of the images in the 
subset by projecting selected 3D feature points into the images in the subset using the 
camera projections of the images in the subset and determining the distance between the 
positions of the projected points and the positions of the corresponding features in the 
images; and 

calculating changed positions for at least some of the selected 3D 
feature points and changed camera projections for the images in the subset by using the 
positions in the images in the subset of the features which correspond to the selected 3D 
feature points to determine changed positions for at least some of the selected 3D feature 
points and changed camera projections for the images in the subset which reduce the 
calculated error, 

wherein the processing means is arranged to process respective 
subsets of images from the sequence sequentially such that each image in the sequence is 
processed in a subset at least once. 

Claim 35 (previously presented): A method of processing data defining 
matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining the 
projection of points on the object surface into the image, the method comprising: 

a selecting step, of using the matching features to select a first image 
in the sequence and a second image in the sequence for each of a plurality of sets of images 
to give initial end images in the sequence for each set; 
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a set forming step, of, for the selected pair of end images for each 
set, using the matching features to calculate a camera projection for one of the images in the 
pair relative to the other image in the pair, to form a set thereform; 

a set expanding step, of increasing the number of images in at least 
some of the sets by using the matching features to calculate a respective camera projection 
for each of at least some of the images lying in the sequence between end images of a set 
and each of at least some of the images lying in the sequence between consecutive sets, 
each camera projection being calculated relative to the images in a set; and 

a set merging step, of using the matching features to determine the 
relationship between the camera projections calculated for the different sets, thereby to 
merge the sets. 

Claim 36 (previously presented): A method according to claim 35, wherein, 
in said selecting step, an image to form an end image is selected in dependence upon the 
number of matching features in the image which are also present in the preceding end 
image in the sequence. 

Claim 37 (previously presented): A method according to claim 35, wherein, 
in said set expanding step, the camera projections for images lying between the end images 
for a given set are calculated before the camera projections for images lying between the 
given set and adjacent sets in the sequence are calculated. 
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Claim 38 (previously presented): A method according to claim 35, wherein, 
in calculating camera projections for images lying between consecutive sets: 

(i) the selected end images are prioritised in dependence upon an 
accuracy of the calculated camera projections of the end images; 

(ii) the end image with the highest priority is selected; 

(iii) the camera projection for the image in the sequence adjacent 
the selected end image is calculated relative to the selected end image to form a new end 
image; 

(iv) the priority of the end images is updated in dependence upon 
the accuracy of the camera projection calculated in step (iii); and 

(v) steps (ii) to (iv) are repeated. 

Claim 39 (previously presented): A method according to claim 35, wherein 
pairs of end images selected in said selecting step are considered in turn and, for a given 
pair of end images, said set forming step to form a set therefrom and said set expanding 
step to calculate a respective camera projection for at least one image lying in the sequence 
between the pair of end images and at least one image lying in the sequence between an end 
image of the pair and an end image for a consecutive set, are performed before performing 
said set forming step and said set expanding step for the next pair of end images to be 
considered. 

Claim 40 (original): A method according to claim 35, further comprising 
the step of generating a signal conveying the calculated camera projections. 
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Claim 41 (original): A method according to claim 40, farther comprising 
the step of recording the signal either directly or indirectly, 

Claim 42 (original): A method according to claim 35, wherein the sequence 
of images is a temporal sequence in which the images are arranged in time-of-recording 
order. 



\ Claim 43 (original); A method according to claim 35 9 wherein the images 

\ comprise video images. 

y 

/ Claim 44 (original): A method according to claim 35, further comprising 

the step of processing image data defining the images in the sequence to generate the data 
defining the matching features. 



Claim 45 (original); A method according to claim 35, further comprising 
the step of using the matching features and the calculated camera projections to generate 
data defining a 3D computer model of the object in the images. 



Claim 46 (original): A method according to claim 45, further comprising 
the step of generating a signal conveying the 3D computer model. 

Claim 47 (original): A method according to claim 46, further comprising 
the step of recording die signal either directly or indirectly. 
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Claim 48 (previously presented): A method of processing data defining 
matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining a projection 
of the object into the image, the method comprising the steps of: 

selecting a plurality of groups of the images, each group comprising 
at least two images and at least one pair of consecutive groups in the sequence having 
images in the sequence therebetween which are not part of a group; 

forming a set of images from each selected group by calculating a 
respective camera projection for each image in the group relative to a reference image in 
the group; 

calculating a respective camera projection for each of at least some 
of the images in the sequence not selected to form a group relative to a reference image in a 
group; and 

calculating the relationship between the camera projections 
calculated for each group. 

Claim 49 (previously presented): An apparatus for processing data defining 
matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining the 
projection of points on the object surface into the image, comprising: 

an image selector for using the matching features to select a first 
image in the sequence and a second image in the sequence for each of a plurality of sets of 
images to give initial end images in the sequence for each set; 
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a set former for using the matching features to calculate, for the 
selected pair of end images for each set, a camera projection for one of the images in the 
pair relative to the other image in the pair, to form a set therefrom; 

a set expander for increasing the number of images in at least some 
of the sets by using the matching features to calculate a respective camera projection for 
each of at least some of the images lying in the sequence between end images of a set and 
each of at least some of the images lying in the sequence between consecutive sets, each 
camera projection being calculated relative to the images in a set; and 

a set merger for using the matching features to determine the 
relationship between the camera projections calculated for the different sets, thereby to 
merge the sets. 

Claim 50 (previously presented): An apparatus according to claim 49, 
wherein said image selector is arranged to select an image to form an end image in 
dependence upon the number of matching features in the image which are also present in 
the preceding end image in the sequence. 

Claim 51 (previously presented); An apparatus according to claim 49, 
wherein said set expander is arranged to calculate the camera projections for images lying 
between the end images for a given set before calculating the camera projections for images 
lying between the given set and adjacent sets in the sequence. 
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Claim 52 (previously presented): An apparatus according to claim 49, 
wherein said set expander is arranged to calculate camera projections for images lying 
between consecutive sets by; 

(i) prioritising the selected end images in dependence upon an 
accuracy of the calculated camera projections of the end images; 

(ii) selecting the end image with the highest priority; 

(iii) calculating the camera projection for the image in the 
sequence adjacent the selected end image relative to the selected end image to form a new 
end image; 

(iv) updating the priority of the end images in dependence upon 
the accuracy of the camera projection calculated in step (iii); and 

(v) repeating steps (ii) to (iv). 

Claim 53 (previously presented); An apparatus according to claims 49, 
wherein the apparatus is arranged to operate so that pairs of end images selected by the 
image selector are considered in turn and, for a given pair of end images, said set former is 
arranged to form a set therefrom and said set expander is arranged to calculate a respective 
camera projection for at least one image lying in the sequence between the pair of end 
images and at least one image lying in the sequence between an end image of the pair and 
an end image for a consecutive set, before forming a set and expanding the set for the next 
pair of end images to be considered. 
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Claim 54 (previously presented): An apparatus according to claim 49, 
wherein the sequence of images is a temporal sequence in which the images are arranged i 
time-of-recording order. 

Claim 55 (previously presented): An apparatus according to claim 49, 
wherein the images comprise video images. 



further comprising a 3D computer model data generator for using the matching features and 
the calculated camera projections to generate data defining a 3D computer model of the 
object in the images. 



matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining a projection 
of the object into the image, comprising: 

an image group calculator for selecting a plurality of groups of the 
images, each group comprising at least two images and at least one pair of consecutive 
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Claim 56 (previously presented): An apparatus according to claim 49, 
further comprising a feature matcher for processing image data defining the images in the 
sequence to generate the data defining the matching features, 



Claim 57 (previously presented): An apparatus according to claim 49, 



Claim 58 (previously presented): An apparatus for processing data defining 
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groups in the sequence having images in the sequence therebetween which are not part of a 
group; 

an image set calculator for forming a set of images from each 
selected group by calculating a respective camera projection for each image in the group 
relative to a reference image in the group; 

a camera projection calculator for calculating a respective camera 
projection for each of at least some of the images in the sequence not selected to form a 
group relative to a reference image in a group; and 

a camera projection relationship calculator for calculating the 
relationship between the camera projections calculated for each group. 

Claim 59 (original): A storage device storing instructions for causing a 
programmable processing apparatus to become operable to perform a method as set out in 
at least one of claims 35 to 48. 

Claim 60 (original): A signal conveying instructions for causing a 
programmable processing apparatus to become operable to perform a method as set out in 
at least one of claims 35 to 48. 

Claim 6 1 (previously presented): In an image processing method for 
processing image data defining a sequence of images of at least one object to generate a 
three-dimensional computer model of the object by matching features in the images, using 
the matching features to determine camera projections defining the projection of the object 
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surface into the images, and using the matching features and the calculated camera 
projections to generate data defining the three-dimensional computer model of the object, 
an improvement comprising performing the step of using the matching features to 
• determine the camera projections by; 

using the matching features to select a first image in the sequence 
and a second image in the sequence for each of a plurality of sets of images to give initial 
end images in the sequence for each set; 

for the selected pair of end images for each set, using the matching 
features to calculate a camera projection for one of the images in the pair relative to the 
other image in the pair, to form a set therefrom; 

increasing the number of images in at least some of the sets by using 
the matching features to calculate a respective camera projection for each of at least some 
of the images lying between pairs of end images in the sequence and each of at least some 
of the images lying between consecutive sets in the sequence, each camera projection being 
calculated relative to the images in a set; and 

using the matching features to determine the relationship between the 
camera projections calculated for the different sets, thereby to merge the sets. 

Claim 62 (original): In an image processing apparatus for processing image 
data defining a sequence of images of at least one object to generate a three-dimensional 
computer model of the object by matching features in the images, using the matching 
features to determine camera projections defining the projection of the object into the 
images, and using the matching features and the calculated camera projections to generate 
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data defining the three-dimensional computer model of the object, a method of performing 
the processing to determine the camera projections by: 

selecting a plurality of groups of the images, each group comprising 
at least two images and at least one pair of consecutive groups in the sequence having 
images in the sequence therebetween which are not part of a group; 

forming a set of images from each selected group by calculating a 
respective camera projection for each image in the group relative to a reference image in 
the group; 

calculating a respective camera projection for each of at least some 
of the images in the sequence not selected to form a group relative to a reference image in a 
group; and 

calculating the relationship b|etween the camera projections 
calculated for each group. 

Claim 63 (previously presented): In an image processing apparatus having a 
processor for processing image data defining a sequence of images of at least one object to 
generate a three-dimensional computer model of the object by matching features in the 
images, using the matching features to determine camera projections defining the projection 
of the object surface into the images, and using the matching features and the calculated 
camera projections to generate data defining the three-dimensional computer model of the 
object, wherein the processor is arranged to use the matching features to determine the 
camera projections by: 
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using the matching features to select a first image in the sequence 
and a second image in the sequence for each of a plurality of sets of images to give initial 
end images in the sequence for each set; 

for the selected pair of end images for each set, using the matching 
features to calculate a camera projection for one of the images in the pair relative to the 
other image in the pair, to form a set therefrom; 

increasing the number of images in at least some of the sets by using 
the matching features to calculate a respective camera projection for each of at least some 
of the images lying between pairs of end images in the sequence and each of at least some 
of the images lying between consecutive sets in the sequence, each camera projection being 
calculated relative to the images in a set; and 

using the matching features to determine the relationship between the 
camera projections calculated for the different sets, thereby to merge the sets. 

Claim 64 (pieviously presented): An apparatus for processing data defining 
matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining the 
projection of points on the object surface into the image, comprising; 

selecting means for using the matching features to select a first 
image in the sequence and a second image in the sequence for each of a plurality of sets of 
images to give initial end images in the sequence for each set; 
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set forming means for using the matching features to calculate, for 
the selected pair of end images for each set, a camera projection for one of the images in the 
pair relative to the other image in the pair, to form a set therefrom; 

set expanding mean$ for increasing the number of images in at least 
some of the sets by using the matching features to calculate a respective camera projection 
for each of at least some of the images lying in the sequence between end images of a set 
and each of at least some of the images lying in the sequence between consecutive sets, 
each camera projection being calculated relative to the images in a set; and 

set merging means for using the matching features to determine the 
relationship between the camera projections calculated for the different sets, thereby to 
merge the sets. 

Claim 65 (previously presented): An apparatus for processing data defining 
matching features in images within a sequence of images of at least one object to determine 
a respective camera projection for each of at least some of the images defining a projection 
of the object into the image* comprising: 

means for selecting a plurality of groups of the images, each group 
comprising at least two images and at least one pair of consecutive groups in the sequence 
having images in the sequence therebetween which are not part of a group; 

means for forming a set of images from each selected group by 
calculating a respective camera projection for each image in the group relative to a 
reference image in the group; 
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means for calculating a respective camera projection for each of at 
least some of the images in the sequence not selected to form a group relative to a reference 
image in a group; and 

means for calculating the relationship between the camera 
projections calculated for each group. 

Claim 66 (previously presented): A method of processing data defining (i) 
the positions of features in a sequence of images of at least one object which represent 
features on the object, (ii) an estimate of a respective camera projection for each image 
defining the projection of points on the object into the image, and (iii) 3D feature points 
comprising estimates of the positions in three-dimensions of features on the object 
represented by the features in the images, to generate output data defining further estimates 
of the camera projections, the method comprising processing respective subsets of the 
images in the sequence by: 

calculating an error for the camera projections of the images in the 
subset by projecting 3D feature points into the images in the subset using the camera 
projections of the images in the subset and determining the distance between the positions 
of the projected points and the positions of the corresponding features in the images; and 

calculating changed 3D positions for at least some of the 3D feature 
points that were projected and changed camera projections for at least some of the images 
in the subsetby using the positions in die images in the subset of the features which 
correspond to the 3D feature points that were projected to detennme changed 3D positions 
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for at least some of the 3D feature points that were projected and changed camera 
projections for at least some of the images in the subset which reduce the calculated error, 

wherein the respective subsets of images are selected sequentially 
from the sequence so that each image in the sequence is processed in a subset at least once. 

Claim 67 (previously presented): An apparatus for processing data defining 
(i) the positions of features in a sequence of images of at least one object which represent 
features on the object, (ii) an estimate of a respective camera projection for each image 
defining the projection of points on the object into the image, and (iii) 3D feature points 
comprising estimates of the positions in three-dimensions of features on the object 
represented by the features in the images, to generate output data defining further estimates 
of the camera projections, comprising a processor operable to process respective subsets of 
the images in the sequence by: 

calculating an error for the camera projections of the images in the 
subset by projecting 3D feature points into the images in the subset using the camera 
projections of the images in the subset and determining the distance between the positions 
of the projected points and the positions of the corresponding features in the images; and 

calculating changed 3D positions for at least some of the 3D feature 
points that were projected and changed camera projections for at least some of the images 
in the subset by using the positions in the images in the subset of the features which 
correspond to the 3D feature points that were projected to determine changed 3D positions 
for at least some of the 3D feature points that were projected and changed camera 
projections for at least some of the images in the subset which reduce the calculated error, 
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wherein the processor is arranged to process respective subsets of 
images from the sequence sequentially such that each image in the sequence is processed ir 
a subset at least once. 

Claim 68 (previously presented): An apparatus for processing data defining 
(i) the positions of features in a sequence of images of at least one object which represent 
features on the object, (il) an estimate of a respective camera projection for each image 
defining the projection of points on the object into the image, and (hi) 3D feature points 
comprising estimates of the positions in three-dimensions of features on the object 
represented by the features in the images, to generate output data defining further estimates 
of the camera projections, comprising processing means for processing respective subsets 
of the images in the sequence by: 

calculating an error for the camera projections of the images in the 
subset by projecting 3D feature points into the images in the subset using the camera 
projections of the images in the subset and determining the distance between the positions 
of the projected points and the positions of the corresponding features in the images; and 

calculating changed 3D positions for at least some of the 3D feature 
points that were projected and changed camera projections for at least some of the images 
in the subset by using the positions in the images in the subset of the features which 
correspond to the 3D feature points that were projected to determine changed 3D positions 
for at least some of the 3D feature points that were projected and changed camera 
projections for at least some of the images in the subset which reduce the calculated error, 
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wherein the processing means is arranged to process respective 
subsets of images from the sequence sequentially such that each image in the sequence is 
processed in a subset at least once. 

Claim 69 (previously presented): A storage device storing computer 
\^ ^ program instructions for programming a programmable processing apparatus to become 
operable to perform a method as set out in claim 66. 

Claim 70 (previously presented): A signal conveying computer program 
instructions for programming a programmable processing apparatus to become operable to 
perform a method as set out in claim 66. 
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